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Limited  time  and  wide  geographical  dispersion  of  both  units 
and  individuals  in  the  National  Guard  and  Army  Reserve~the  Re¬ 
serve  Component  (RC) — make  it  difficult  and  costly  for  soldiers 
to  travel  to  brancli  schools  for  training.  Therefore,  the  RC  is 
exploring  alternatives  that  will  use  technology  to  bring  training 
and  educational  opportunities  to  the  soldiers'  homes.  One  of 
these  alternatives  establishes  remotely  conducted  classes  in 
which  individuals  are  linked  with  each  other  and  their  instruc¬ 
tors  using  asynchronous  computer  conferencing  (ACC) . 


This  report  summarizes  the  findings  on  the  effectiveness  and 
Costs  of  using  asynchronous  computer  conferencing  for  RC  distrib¬ 
uted  training.  The  report  was  developed  by  the  U.S.  Army  Re¬ 
search  Institute  for  the  Behavioral  and  Social  Sciences  (ARl) 
Boise  Element  within  the  charter  of  the  Training  Technology  Field 
Activity,  Gowen  Field,  whose  mission  is  to  improve  Reserve  Compo¬ 
nent  training  effectiveness  and  efficiency  through  the  testing 
and  application  of  technology.  The  research  task  supporting  this 
mission.  Application  of  Technology  to  Meet  Reserve  Component 
Needs,  is  organized  under  the  Training  for  Combat  Effectiveness 
progr2tm  area.  The  National  Guard  Bureau,  Forces  Command,  and 
Training  and  Doctrine  Command  Headquarters  (TRADOC  HQ)  sponsored 
this  project  under  tlie  Memorandum  of  Understanding  signed  12  June 
1985  that  established  the  office.  Project  results  have  been 
briefed  to  TRADOC  HQ,  Forces  Command,  Office  of  the  Chief,  Army 
Reserve,  and  the  National  Guard  Bureau. 
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COST  AND  EFFECTIVENESS  OF  HOME  STUDY  USING  ASYNCHRONOUS  COMPUTER 
CONFERENCING  FOR  RESERVE  COMPCNENT  TRAINING 


EXECUTIVE  SUMMARY 


Requirement: 

This  report  evaluates  the  costs  and  effectiveness  of  using 
asynchronous  computer  conferencing  (ACC)  techniques  to  provide 
high-quality  remotely  delivered  training  to  the  U.S.  Army  Reserve 
Component  (RC) . 


Procedure : 

The  evaluation  used  a  portion  of  the  Engineer  Officer  Ad¬ 
vanced  Covirse  (EOAC)  as  a  test  bed.  Coturse  materials  that  taught 
the  same  content  presented  in  the  resident  course  were  developed 
for  remote,  asynchronous  presentation.  Media  used  were  paper  and 
computer-based  instruction,  video  tapes,  and  computer  conferenc¬ 
ing.  The  efficacy  of  remote  training  was  compared  with  that  of 
the  resident  progzam  for  throughput,  performance,  and  cost. 


Findings: 

Comparisons  of  home  study  by  ACC  and  resident  training 
showed  that  (1)  ACC  training  costs  less,  (2)  there  were  no 
differences  in  test  scores,  (3)  ACC  studbnts  perceived  thf^.y 
learned  more  from  the  course,  and  (4)  more  resident  students 
completed  the  course. 


Utilization  of  Findings: 

Based  on  these  data,  it  appears  that  ACC  courses  can  be  a 
cost-effective  alternative  to  resident  training.  These  findings 
support  the  efficacy  of  distributed  training  for  the  RC  and  con¬ 
tribute  to  the  development  of  Total  Force  distributed-training 
strategies  and  decisions  concerning  the  implementation  of  new 
training  technologies. 
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COST  AND  EFFECTIVENESS  OF  HOME  STtJDY  USING 
ASYNCHRONOUS  COMPUTER  CONFERENCING 
FOR  RESERVE  COMPONENT  TRAINING 

Background 

Geographical  dispersion,  limited  resources  and  civilian 
job  and  family  demands  make  travel  to  distant  locations  for 
training  and  education  difficult  for  members  of  the  Reserve 
Components  (RC) .  The  Army  RC,  composed  of  the  United  States 
Army  Reserve  (USAR)  and  the  Army  National  Guard  (ARNG) , 
consists  of  approximately  7000  units  scattered  across  the 
nation  and  overseas  at  more  than  4000  armories  and  reserve 
centers.  The  typical  RC  training  year  includes  only  39 
days  traditionally  distributed  over  12  weekends  and  one  15~ 
day  annual  training  session  during  which  units  must  accomplish 
both  individual  and  collective  training  requirements.  Thus,  when 
individuals  must  attend  training  or  educational  courses  away  from 
home,  they  are  absent  from  their  units,  detracting  from  unit 
collective  training  and  the  organization's  ability  to  attain 
wartime  readiness  levels. 

In  addition  to  hardsn_;?s  for  soldiers  and  units, 
training  in  residence  at  active  component  branch  schools  is 
expensive.  In  a  plan  to  reduce  this  cost,  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC)  has  proposed  a  50%  cut 
in  resident  training  time  by  the  year  2007  (TRADOC  PAM  350-4, 
1989) .  Similar  reductions  may  be  forthcoming  for  Reserve  Forces 
(RF)  Schools  and  National  Guard  state  academies. 

The  pu^ose  of  this  report  is  to  summarize  an 
investigation  into  an  alternate  means  for  meeting  the 
training/educational  needs  of  the  RC  called  Asynchronous 
Computer  Conferencing  (ACC).  In  general,  ACC  is  a  means  of 
communicating  from  different  locations  at  different  times 
(i.e.,  asynchronously)  using  a  computer  network.  An  "electronic 
classroom"  is  established  by  connecting  students  with  one  another 
and  the  instructional  staff.  A  student  or  instructor  can 
participate  in  classroom  activities  from  any  location  using 
existing  telephone  lines  and  a  computer  equipped  with  a  modem. 
Computer  conferencing  is  similar  to  electronic  mail,  allowing 
students  to  work  together  in  groups,  ask  questions  of 
instructors,  tutor  classmates  or  just  share  thoughts  and 
experiences.  It  can  also  enable  the  instructional  staff  to 
direct  small  group  instruction,  ask  and  answer  questions, 
test,  provide  timely  feedback  and  give  remedial  instruction. 

The  goals  of  the  present  research  were  to  (a)  develop  and 
test  a  new  training  option  using  ACC  that  would  not  require 
soldiers  to  leave  their  homes  and  units,  but  would  maintain  the 
quality  of  training  found  at  tho  branch  school  and  (b)  determina 
the  cost  and  effectiveness  of  developing  and  operating  the  ACC 
alternative. 
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The  ACC  concept  for  conducting  RC  military  education 
was  evaluated  using  the  Engineer  Officer  Advanced  Course 
(EOAC)  as  a  test  case.  An  officer  aav *nced  course  was 
chosen  because  noncompletion  of  OAC  has  been  identified 
historically  as  the  primary  reason  why  RC  captains  fail  to  be 
selected  for  promotion.  In  FY  89,  for  example,  only  59%  of  those 
eligible  had  completed  their  advanced  course  resulting  in  an 
overall  promotion-to-major  selection  rate  of  only  50.5%  (official 
Army  Reserve  Personnel  Center  data).  The  dropout  problem  can  be 
attributed  to  several  factors  including  the  cost  of  sending  RC 
soldiers  to  resident  schools,  the  limited  time  available  to 
attend  resident  training  because  of  civilian  and  military 
reserve  responsibilities  and  quality/motivational 
inconsistencies  found  in  current  non-resident  training. 

EOAC  was  selected,  in  particular,  because  approximately 
72%  of  the  Army's  engineer  assets  are  RC;  the  content  of  the 
EOAC  is  a  mix  of  technical,  tactical  and  problem  solving 
tasks;  and  because  EOAC  consists  largely  of  small  group 
instraction  (SGI) ,  presenting  a  major  challenge  to  those 
executing  non-resident  training. 

Method 

Participants 

Fourteen  RC  captains  and  lieutenants  began  the  computer 
mediated  home-study  course.  Six  were  members  of  the  USAR  and 
eight  were  ARNG.  All  but  one  were  mole.  The  instructional  staff 
consisted  of  a  civilian  full-time  course  manager/administrator 
responsible  for  the  overall  operation  of  the  course  and 
supervision  of  supporting  staff;  he  was  also  a  member  of  the 
USAR.  The  supporting  staff  consisted  of  four  part-time 
instructors, . all  members  of  the  USAR;  their  responsibilities 
included  monitoring  student  progress,  direccing  group  discussions, 
and  conducting  remedial  instruction. 

For  comparison  purposes,  course  performance  data  were 
collected  from  339  RC  officers  taking  EOAC  in  residence  at 
USAES  during  FY  87,  88  and  89.  Demog.aphic  information  was  also 
collected  from  49  RC  students  in  residence  during  FY  89. 

Course  Description 

Module  6  of  the  seven  module  (each  two  weeks  long)  United 
States  Army  Engineer  School  resident  EOAC  course,  consisting  of 
66  program  of  instruction  (POI)  hours,  was  taught.  The  POI  was 
identical  for  the  resident  and  ACC  courses.  The  nine  topics 
covered  were:  Airfield  Damage  Repaix*  (2  hrs.).  Military 
Petroleum  Pipelines  (4  hrs.).  Asphalt  Production  (3  hrs.). 
Flexible  Pavement  Structures  (5  hrs.).  Roads  and  Airfields  (7 
hrs.).  Military  Bridges  (24  hrs.).  Rear  Operations  (2  hrs.},  a: 
video  presentation  of  a  talk  by  an  allied  officer  (1  hrs.),  and  a 
writing  requirement  (2  hrs.).  In  a  course  culminating  practical 


exercise  called  a  "Capstone”  (1'.  hrs.)/  students  worked  in  teams 
of  five  or  six  and  were  required  to  integrate  technical 
information  learned  throughout  the  module  into  plans  for  carrying 
out  construction  directives  issued  by  a  notional  higher 
headquarters.  The  Capstone  scenario  placed  the  teams  as  staff 
groups  in  a  simulated  combat  environment  in  which  they  were  to 
develop  plans  appropriate  to  this  scenario  and  resource 
constraints;  each  team  then  briefed  its  plan  to  an  actual  field 
grade  officer  poking  as  the  commander  using  computer 
conferencing. 

The  resident  course  was  presented  in  a  traditional  military 
classroom  setting  with  students  living  in  residence  for  the 
duration  of  the  course,  attending  classes  each  day,  five  days  a 
week.  In  addition,  students  were  expected  to  study  during 
weekends  and  evenings.  Typical  media  included  lectures, 
computer-aided  instruction  and  small  group  instruction  with 
practical  exercises.  Testing  consisted  of  a  final  exam  and  a 
graded  Capstone  exercise. 

The  materials  used  in  the  resident  course  were 
converted  into  a  form  suitable  for  ACC  (Hahn,  Harbour,  Wells, 
Schurman,  and  Daveline,  1990) .  The  media  selected  for  ACC 
were  determined  by  both  the  availability  of  the  equipment  at 
the  students'  homes  and  the  suitability  for  meeting  the 
learning  objectives.  Figure  1  compares  the  methods  and  media 
used  in  the  resident  course  with  the  ACC-based  course.  Table  1 
shows  the  percentage  of  course  hours  devoted  to  particular 
media/tasks. 


Table  1 

Media/Task  Percentages 


Media/Task  .  . 

% 

Computer-Based  Instruction 

19% 

Team  Synchronous 

4% 

T( am  Asynchronous 

20% 

Video 

2% 

Print 

41% 

Quiz/Exam/Raview 

14% 

RESIDENT  COURSE 


ACC  COURSE 


Live  Lecture 


Paper-based  readings 
Storyboards 
Author  CAI  Lessons 
Videotape  of  Lecture 


Live  Exam  Review 

Live  Proctored  Exam 


Lead-through  Prac  Exercises 


Smal  Group  Practical  Exercises - ► 


Homework 


Videotape 


.^synchronous  Discussion  Ex^  Review 

Computer  Proctored  Exam 

Author  CAI  Lessons 
Spreadsheet  exercises 
Paper-based  exercises 

Irtcfividual  Problem  Solving 
Synchrorjous  Group  Problem  Solving 
Asynchronous  Group  Problem  Solving 


Homework 

Video 

Transcript  of  Videotape 


ReacSngs  . . . . . . Readings 

PtatoCAl  - - - - -  PlatoCAl 

Live  Role  Play  '  - - - - - - - -  Videotaped  Role  Ray 

Writing  Assignments  - - - Writing  Assignment 

Live  Capstone  . . . . . . . — - ik.  Capstone  via  Asynchronous  Meetings 

Figure  1.  Methods  used  in  resident  course  with  corresponding 
methods  appropriate  for  the  Asynchronous  Computer  Conferencing 
course. 


EaulPinent.  Cororounications  an«l  _Sja£tvare 

Each  student  was  provided  a  personal  computer  with  a  20 
megabyte  hard  disk,  keyboard,  color  monitor  and  printer. 

Software  and  courseware  loaded  on  each  computer  consisted  of 
(a)  the  specially  developed  course  management  system  and 
communications  package;  (b)  computer-assisted  instruction  and 
tests;  (c)  a  word  processing  package;  and  (d)  a  spreadsheet 
package . 

A  communications  system  for  ACC  was  provided  through  Army 
Forum,  a  division  of  the  Office  of  the  Director  of  the  Army 
Staff.  The  host  computer  was  located  at  Wayne  State  University 
and  used  Confer  II  [Copyright  (c)  1986  by  Advertel  Communication 
Systems,  Inc.]  software. 

Specialized  Software  was  designed  to  create  an  interface  . 
between  Confer  II  and  the  student.  It  was  designed  to  mimic 
the  functions  performed  at  the  resident  school  and  to  help 
create  the  concept  of  the  electronic  classroom.  Major  functions 
the  students  Would  perform  were  assigned  to  "rooms",  as  shown  in 
Figure  2.  This  schoolhouse  metaphor  served  to  aid  the  ACC 
student  in  understanding  the  ACC  learning  environment.  Key  rooms 
included  (a)  a  room  for  administrative  matters  such  as 
announcements  and  class  standings  (the  office) ;  (b)  rooms  for 
private  communications  (mail  room,  instructor  help  room,  team 
leader  help  room) ;  (c)  rooms  for  group  communication  (break  room, 
class  room,  team  rooms) ;  (d)  a  room  to  access  lessons  on  their 
hard  disks  (the  classroom) ;  and  (e)  rooms  to  do  homework  and 
other  written  assignments  (the  writing  center) .  In  addition,  a 
task  list  was  displayed  which  guided  the  student  through  the 
learning  activities  and  showed  personal  progress.  Timed  quiz  and 
exam  questions  were  presented,  scored  and  reported 
electronically.  Test  security  was  maintained  by  denying  students 
any  opportunity  to  either  print  tests  or  to  recall  a  test  to  the 
computer  screen  once  time  expired  for  the  test. 


Writing 

Room 

Classroom 

Break  Room 

Team  Rooms 

Learning 
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Nail  Room 

Office 

Team  Leader 
Help  Room 

Task  List 

Instructor 
Help  Room 

Figure  2.  The  software  Interface  using  the  schoolhouse  metaphor. 
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Procedure 


Computer  Assembly  and  Training.  Students  were  mailed  all 
computer  ec^ipment  and  course  materials  with  written  instructions 
on  assembling/ operating  their  computers  and  software.  Also 
provided  was  a  toll  free  "hot  line"  telephone  number  for 
resolving  hardware/software  problems.  In  addition,  the  students 
received  precourse  training  materials  to  familiarize  them  with 
the  operation  of  the  computer  and  software. 


Part-time  instructional  staff  were  provided  the  same 
type  of  equipment  and  software  as  the  students.  Additionally, 
they  attended  a  five  day  training  course  on  instructional 
responsibilities,  teaching/motivational  techniques  and  operation 
of  hardware/software.  Instructional  staff  and  researchers  met  in 
person  for  this  training  which  combined  lecture  and  hands-on 
practice  with  the  computer. 


Course  Progression.  Students  signed  a  contract  agreeing  to 
spend  eight  hours  per  week  on  course  materials  and  to  log-on  to 
their  computers  twice  each  week.  As  shown  in  Figure  3,  the 
course  was  executed  in  three  phases,  beginning  in  September,  1988 
and  ending  in  April,  1989.  For  the  first  phase,  students 
completed  assignments  and  tests  at  their  own  pace  using  materials 
designed  for  individual  instruction.  They  communicated  with  the 
instructional  staff,  but  did  not  participate  in  group  exercises 
with  other  students.  Grade  points  for  all  assignments  and  tests 
were  awarded  individually.  In  the  second  phase,  self-pacing  was 
eliminated  and  all  srudents  were  given  a  fixed  schedule  of 
assignments  and  tests.  The  students  were  also  organized  into  two 
teams  and,  in  addition  to  individual  assignments,  completed  group 
tasks  for  which  all  members  of  the  team  received  the  same  grade. 
In  the  third  phase,  each  team  completed  the  Capstone  practical 
exercise  and  presented  their  solution  in  a  formal  briefing  held 
synchronously  on  their  computers  to  a  field  grade  officer  from 
the  support  staff  who  rolepplayed  their  engineer  brigade/group 
commander. 


Phase  1 


Phase  2 


Phase  3 


Individual 

Self-Pacing 

Instiruction 


^mall  Group 
iPacing  and 
[instruction 


Figure  3.  The  three  phases! 
Conferencing  course. 
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Data  Analysis 


Four  types  of  data  were  collected:  (a)  test  and  homework 
scores;  (b)  student  perceptions  of  how  much  they  thought  they 
had  learned;  (c)  course  completion  rates;  and  (d)  costs  of 
development/conversion  and  execution.  Because  there  were 
several  dependent  measures,  comparisons  of  the  resident  to 
the  ACC  course  were  made  using  multivariate  analysis  procedures 
for  a  two-group  design  (Harris,  1975). 

Test,  homework  and  practical  exercise  scores.  Tests  and 
homework  assignments  were  the  same  as  those  used  for  the  USAES 
resident  Module  6.  Multiple-choice  tests  were  administered  after 
each  topic  and  immediately  scored  by  the  computer.  Students  were 
not  allowed  to  take  additional  tests  until  they  passed  the 
preceding  test.  Homework  was  graded  by  the  instructor  and 
returned  with  comments,  usually  within  24  hours  of  receipt.  It 
should  be  noted  that  in  the  USAES  resident  course,  a 
comprehensive  module  exam  is  administered  at  the  end  of  the 
two  week  block,  whereas  for  the  ACC  students,  separate  tests 
were  administered  after  each  topic. 

Student  perceptions.  Both  the  resident  and  ACC  groups 
were  administered  surveys  before  and  after  the  course  to 
determine  student  self-assessment  of  knowledge  on  the  course 
topics  (e.g.,  how  much  they  thought  they  learned).  Students 
rated  their  perceived  amount  of  knowledge  on  engineer 
technical  and  tactical  skills  on  a  five  point  scale.  A 
measure  of  the  perceived  amount  learned  was  derived  by 
computing  the  difference  between  ratings  before  and  after  the 
course  for  each  student. 

Completion  rate.  The  percentage  of  students  who 
completed  the  resident  and  the  ACC  courses  was  the  completion 
measure.  Any  ACC  student  who  failed  to  complete  the  course 
in  its  entirety  was  administered  a  survey  to  determine  the 
reason (s)  for  noncompletion. 

Cost.  The  costs  in  time  and  dollars  to  develop/ convert 
course  materials  and  run  one  iteration  of  the  66  POI  hour 
module  were  computed.  These  estimates  were  based  on 
assumptions  that  (a)  the  data  gathered  from  the  66  POI  hours 
converted  and  tested  in  this  research  will  generalize  to 
other  EOAC  modules;  (b)  previously  developed  software, 
communications  packages,  and  instructor  training  courses 
(Harbour,  Daveline,  Hells,  Schurman,  Hahn,  1990)  will  be 
used;  and  (c)  the  guidelines  for  course  development/ conversion 
summarized  later  in  Table  3  reflect  an  accurate  level  of  effort. 

The  conversion  estimates  are  based  on  a  TRADOC 
publication  entitled  "Estimated  Time  Values  for  Training 
Developments  ( 1984 ) "  produced  by  the  TRADOC  Management 
Engineering  Activity  (TRAMEA) .  Fstimated  Time  Values  (ETVs) 
are  expressed  as  sCaff  hours  and  listed  in  categories  and 
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sxib-categories  within  the  TRAMEA  docvunent.  Modified  category 
names  have  been  used  in  this  report  to  better  describe  the 
processes  actually  undertaken  in  the  present  research.  The 
development/conversion  staff  hours  reflect  an  estimate  based  on  a 
matchup  of  actual  time  spent  by  the  ACC  staff  with  the  ETV 
guidelines.  An  extrapolation  from  the  ETVs  to  actual  dollar 
costs  was  made  based  on  estimates  of  contract  and  in-house  ^ourly 
rates  for  developing/converting  a  resident  course  to  ACC.  The 
contract  rate  was  based  on  the  actual  time  and  cost  spent  in 
developing/converting  the  ACC  course.  An  in-house  rate  was 
derived  from  pay,  allowances,  benefits  and  base  operations  costs 
as  detailed  in  Appendix  B. 

The  costs  to  execute  the  66  POI  hours  were  estimated  for 
each  of  the  categories  listed  in  the  left  column  of  Table  4. 

The  start-up  and  recurring  costs  for  each  course  iteration  of 
50  students  were  computed  separately.  It  is  assumed  that 
start-up  costs  would  be  repeated  every  five  iterations.  In 
addition,  estimates  were  made  of  projected  costs  if  the  training 
were  executed  completely  by  in-house  personnel  versus  totally 
under  contract,  based  again  on  the  hourly  rates  previously 
discussed. 

In  addition,  cost  projections  for  conducting  and 
executing  the  entire  EOAC  (seven  modules)  using  ACC  were 
computed.  This  projection  was  based  on  the  actual  cost  of 
the  single  Module  6  adjusted  for  the  cost  of  more  portable 
equipment  (i.e.,  lap-top  computers) . 

Costs  of  training  EOAC  students  in  residence  at  USAES 
were  obtained  from  a  report  (ATRM-159)  produced  by  the  Office 
of  the  Deputy  Chief  of  Staff  for  Resource  Management  (DCSRM) , 
Headquarters,  TRADOC.  These  costs  are  detailed  at  Appendix  0. 

Results 

Test.  Homework  and  Practical  Exercise  Scores 

As  shown  at  the  top  of  Table  2,  there  were  no  reliable 
differences  between  the  test,  homework  or  practical  exercise 
scores  of  students  in  residence'  versus  ACC.  (Demographically , 
the  resident  group  did  not  d.ffer  significantly  from  the  ACC 
group) . 

Student.  , Percept  ions 

The  ACC  students  showed  significantly  greater  gains  in 
terms  of  how  much  they  reported  they  learned  during  the 
course  [Milks  Lambda  »  0.49,  F{15,28)  •  1.96,  p<0.05]  as  detailed 
in  Appendix  A.  Another  perception,  shared  by  all  the  students, 
was  that  eight  hours  per  week  was  a  reasonable  amount  of  time  to 
spend  on  an  ACC  course,  but  those  eight  hours  must  include  the 
administrative  time  required  for  upload,  download,  etc. 
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CoaiPletion  Rate 

Historically,  the  completion  rate  for  resident  EOAC 
students  is  95%  or  better.  However,  only  64%  of  the  students 
in  the  experimental  ACC  class  finished  the  entire  course. 

The  reasons  most  frequently  cited  for  ACC  noncompletion 
included  conflicts  due  to  family  and  job  commitments  and 
delays  due  to  changes  in  the  experimental  course  schedule. 

Cost:  One  EOAC  Module  of  66  POI  Hours 

Course  conversion.  A  projection  of  the  total  ETVs  needed 
to  convert  the  66  POI  hours  of  Module  6,  as  it  was  configured 
at  the  time  of  the  research,  is  4250  staff  hours  as  shown  in 
Table  2  and  detailed  in  Appendix  B.  If  the  conversion  were 
accomplished  completely  by  in-house  staff,  the  cost  would  be 
$152,400  (based  on  the  rates  shown  in  Appendix  C) .  If  the 
conversion  were  done  under  contract,  the  cost  is  estimated  at 
$266,700.  In  a  further  comparison,  the  estimated  cost  for  full 
development  of  the  same  66  POI  hours  of  resident  training  (ETV 
equal  to  5881  staff  hours)  is  $187,700  at  the  in-house  rate. 


Table  2 

Estimated  Time  Values  for  Asynchronous  Computer  Conferencing 
Conversion  of  66  Program  of  Instruction  Hours 


Category 

Staff  Hours 

%  Effort 

Course  Requirements  Analysis 

435 

10% 

Course  Design 

163 

4% 

Course  Development 

2589 

61% 

CBI  /  Slide  Conversion* 

812 

19% 

Video  Tape  Production 

251 

6% 

' 

TOTAL  4250 

■ 

'includes  the  hours  to  prepare  and  supervise  a  contract  for 
CBI  development  plus  the  hours  spent  to  actually  perform  the 
low  level  CBI  conversion  required  by  the  ACC  Project. 


Execution.  In  Table  3  are  listed  both  the  estimated 
start-up  and  recurting  costs  of  executing  a  66  hour  ACC 
course  for  50  students. 

Table  3 

Asynchronous  Computer  Conferencing  Execution  Costs  f$K) 
within-Government/Contractor 


Category 

Start-up 

In-House/Cont ract 

Recurring 

In-House/Contract 

Production 

16.8/16.8 

Equ  ^  X"‘ment 

93.5/93.5 

Tr’'  nn 

23.8/46.7 

0.3/0. 6 

8. 0/8.0 

Opera, 

and  Support  — 

96.1/189.0 

Totals 

117.3/140.2 

121.2/214.4 

Amortized  total  costs.  Figure  3  shows  how  costs  would  be 
amortized  over  10  Module  6  iterations.  ACC  (in-house)  becomes 
less  costly  than  resident  after  two  iterations.  ACC  (contract) 
require^  about  seven  iterations  to  match  the  cost  of  the  resident 
option,  but  then  steadily  increases  savings  over  residence 
thereafter. 
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Figure  4.  Cumulative  costs  of  course  conversion  and  execution 
for  Asynchronous  Computer  Conferencing  and  resident  training. 


Cost;  Projections  for  an  Entire  E0AC~RC  of  Seven  Modules 

As  indicated  earlier,  an  assumption  has  been  made  that 
Module  6  approximates  the  other  six  resident  EOAC-RC  modules 
in  terms  of  duration  and  type  of  material  covered.  At  the 
Engineer  School,  two  weeks  are  devoted  to  each  module.  Each 
ACC  module  will  require  about  five  months  to  complete 
extending  an  entire  seven  module  course  to  a  minimum  of  35 
months  if  modules  were  delivered  back-to-back.  Interviews 
and  surveys  of  ACCi  experimental  course  students  indicate  that 
back-to-back  modules  would  adversely  affect  course  retention. 
Therefore,  for  the  purposes  of  this  model,  a  course  duration 
of  3  1/2  years  (42  months)  has  been  assumed  which  provides  for 
breaks  between  modules.  The  projection  is  based  on  two 
iterations  of  this  3  1/2  year  course  for  seven  years. 

The  instructional  staff  would  consist  of  one  full-time 
instructor  plus  one  half-time  assistant  or  some  combination 
of  part-time  subject  matter  experts.  Ideally,  the  full-time 
instructor  would  remain  with  the  class  from  start  to  finish 
and  would  be  based  at  the  branch  school,  collocated  with 
assistant (s)  and  any  other  administrative  staff  (e.g.,  a 
course  coordinator/ supervisor,  course  developers) .  A  support 
element  would  also  be  required  to  handle  logistics.  Each 
operating  course  would  require  the  part  time  assistance  of  an 
individual  to  coordinate  the  production  of  materials 
and  the  shipment  of  those  materials  and  computer  hardware  to 
the  students. 

For  this  example,  it  is  assumed  that  students  and 
instructors  would  keep  their  equipment  for  an  entire  cycle 
and  that  the  machines  would  last  through  two  complete  cycles 
(seven  years)  before  replacement.  It  is  also  assiuned  that 
instructors  would  not  require  significant  retraining  prior  to 
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the  end  of  a  complete  cycle.  In  addition,  the  example  uses 
contract  rates  for  conversion  and  in-house  rates  for 
execution. 

A  student  class  size  of  50  is  used  as  a  best  estimate  of 
the  number  of  students  per  ACC  class  that  would  be 
practicable  under  the  implementation  model  used.  Two  hundred 
fifty  students  per  year  are  shown  for  residence  based  on  FY 
89  EOAC-RC  attendance  figures  from  the  Army  Training 
Requirements  and  Resources  System  (ATIUIS)  data  base.  Total 
ACC  and  resident  graduates  are  based  on  completion  rates  of 
64%  and  95%,  respectively.  The  ACC  figures  shown  in  Table  5 
assxime  that  every  student  among  the  non-finishers  is  on  hand 
from  beginning  to  end ,  each  thereby  costing  as  much  as  a 
finisher.  The  time  frame  chosen  fox  the  comparison  is  seven 
years,  or  two  iterations  of  the  three  and  a  half  year  ACC 
cycle . 

The  cost  of  graduating  1,662  students  based  on  completion 
rate,  cost  per  student,  and  number  of  students  trained  each  year 
over  a  seven  year  period  is  computed  for  both  resident  and  ACC 
training  options.  In  addition,  the  costs  of  equipment  and  course 
conversion  are  added  for  the  ACC  option.  As  shown  at  the  bottom 
of  Table  4,  there  is  a  projected  savings  of  33%  if  ACC  is  adopted 
in  place  of  resident  training. 

Table  4 

Potential  Long  Term  Savings  ($M) 


Course  Resident  Cost  ($70. 9M)*  -  ACC  Cost  ($47. SM)*^  »  $23.  IM 

Savings 

Percentage  Savings  $23.1M/$70.9M  «  33% 


"Resident  Cost  is  determined  by:  $40. 5K  (per  student)  x  250 
(students  per  year)  x  7  years  costs  $70. 9M  to  provide  1,662 
graduates,  assvuning  95%  completion. 

^ACC  Cost  is  determined  by:  $17. 7K  (per  student)  x  2,597 
(students  over  7  years)  costs  $46.0M  to  „»rovide  1,662  graduates, 
asstiming  64%  completion.  The  $46. OM  plus  a  one  time  course 
conversion  cost  of  $1.8M  results  in  a  total  cost  of  $47. 8M. 
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Discussion 


ACC  appears  to  meet  the  need  of  the  RC  to  complete 
educational  requirements  from  the  home  or  homestation 
without  extended  absences  from  the  unit.  The  *' electronic 
classroom"  could  be  conducted  remotely  from  existing 
educational  institutions  such  as  branch  schools  or  RF 
Schools/State  Academies  in  order  to  maintain  standardized 
instruction  and  relieve  local  commanders  of  educational 
responsibilities.  Instruction  has  the  potential  to  he  of  higher 
quality  and  more  motivating  than  that  found  in  typical 
correspondence  courses.  It  would  consist  of  computer  based 
instruction  supplemented  by  pa^er-based  and  audio/video  materials 
for  both  initial  learning  and  remediation.  In  addition, 
computerized  testi.ig  could  be  conducted  with  the  advantages 
of  test  security,  automatic  timing  and  scoring,  and  immediate 
repoirting  of  results  to  students  and  instructors.  And  unlike 
correspondence  courses ,  students  would  have  ready  access  to  their 
classmates  and  instructor  for  learning  and  motivational  support. 

Students  taking  the  home  study  ACC  course  achieved  test 
scores  equivalent  to  their  resident  counterparts  taught  in  a 
traditional  educational  setting.  Moreover,  the  ACC  soldiers 
experienced  similar  classroom  experiences  such  as  working  in 
small  groups  to  solve  problems,  tutoring  their  peers,  organizing 
and  leading  team  practical  exercises,  and  giving  briefings.  They 
also  experienced  the  same  stresses  stemming  from  meeting 
deadlines  and  being  placed  in  leadership  roles  as  did  their 
resident  counterparts,  even  though  the  asynchronous  aspect  of  ACC 
allowed  work  to  be  spread  out  over  longer  periods.  ACC  students 
also  reported  a  higher  perceived  level  of  learning  during  their 
course  than  did  the  resident  control  group. 

ACC  is  less  costly  than  sending  soldiers  to  resident  schools 
to  meet  their  military  educational  requirements.  It  is  estimated 
that  for  every  five  iterations  of  one  module  of  the  Engineer 
Officer  Advanced  Course,  more  than  $500,000  could  be  saved 
depending  upon  whether  the  course  development/conversion  and 
implementation  tasks  are  accomplished  using  government  or 
contractor  resources.  A  combined  approach,  similar  to  the  one 
described  in  Table  5,  is  a  likely  possibility  with  a  contractor 
used  for  the  conversion  effort  and  goveinnment  assets  used  in 
execution. 

Individual  categories  of  ACC  costs  reveal  that  the  cost 
of  course  conversion  is  highly  significant,  greater  than 
ei^er  equipment  or  ope 'at ions/support.  Total  costs  for  the 
latter  two  categories  would  rise  as  tho  numbers  of  students 
and  instructors  increase.  Both  costs  should  decrease  with 
advances  in  technology.  It  is  important  to  note  that  ACC 
does  not  eliminate  the  requirement  for  instructors  and, 
therefore,  their  salaries.  To  the  contrary,  ready  access  to 
an  instructor  is  key  to  the  concept  of  the  "electronic 
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classroom"  and  provides  that  interactive  link  between  the 
student  an  ‘  the  institution  missing  from  a  stand-alone 
system . 

Dropout  is  a  characteristic  that  adversely  impacts  the 
cost-effectiveness  of  any  course.  The  attrition  rates  for 
ACC  and  the  resident  EOAC-RC  course  were  factored  into  the 
calculations  comparing  their  relative  costs.  Despite  higher  ACC 
attrition,  ACC  was  still  less  expensive.  The  dropout  rate  for  a 
fully  implemented  ACC  course  will  probably  be  much  less  than  the 
experimental  course  because  of  improved  efficiencies. 

Unfo.  tunately,  surveys  of  dropouts  revealed  that  personal 
considerations  were  the  most  significant  factors  influencing  a 
decision  to  guit  the  course  and  those  tend  to  be  the  most 
difficult  to  overcome.  ACC's  longer  duration  increases  the 
chances  for  such  problems  to  develop.  The  ACC  dropout  rate, 
then,  can  probably  be  improved,  but  not  so  much  that  it  will 
match  the  resident  rate. 

ACC  fdr  home  study  may  also  contribute  to  improving 
retention  of  RC  soldiers.  Geographical  isolation,  coupled 
with  the  perception  that  their  time  is  often  ill  used  during 
training,  are  two  factors  that  alienate  soldiers  and  increase 
their  likelihood  of  leaving  the  RC  (Eisley  &  Viner,  1989) . 

ACC  students  seem  to  gain  increased  feelings  of  comradery, 
cohesion,  and  connectedness  with  their  peers.  ACC  home  stUL.y 
should  help  to  counter  feelings  of  isolation  as  well  as  to 
provide  a  sense  of  accomplishment  even  greater  than  that 
resulting  from  completion  of  resident  training.  A  part  of  that 
sense  of  accomplishment  may  stem  from  what  may  be  for  many 
soldiers  an  introduction  to  computer  technologies  and  an 
opportunity  for  others  already  familiar  with  computers  to  sharpen 
their  skills. 
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Rear  Operations  0.136  (1.13)  1.125  (0.64) 
Airfield  Damage  Repair  0.409  (1.13)  1.375  (0.83) 
Pipelines  0.452  (1.13)  1.500  (1.60) 
Asphalt  Production  1.250  (1.08)  1.875  (0.99) 
Flexible  Pavements  0.932  (1.07)  2.125  (0.99) 
Bridging  0.500  (1.08)  0.625  (0.92) 
Roads  and  Airfields  0.809  (0.99)  2.250  (1.17) 


Note:  Scores  were  obtained  by  subtracting  pre-course  ratings 
from  post-course  ratings.  Ratings  were  based  on  a  1-5 
scale. 


Appendix  B 


Estironted  Time  Values  for  ACC  Conversion 


Time  fhrs) 

%  of  Effort 

Course  Recmirements  Analysis 

10% 

Conduct  Initial  Review 

Hi 

Assemble  Draft  Training  Program 
Package 

180 

Coordinate  Draft  Training 

Program  Approval 

42 

Provide  Draft  Training  Program 
Implementation  Assistance 

102 

Course  Design 

4% 

Select  Instructional  Setting 

46 

Identify  Learning  Objective 
Relationship 

20 

Group  Learning  Objectives 

15 

Prepare  Practical  Exercise  Test 

82 

Course  Development 

61% 

Develop  Draft  Course  Management 
Plan 

101 

' 

Revise  Draft  Course  Management 

Plan 

19 

' 

Develop  New  Lesson  Plan 

459 

Develop  New  Student  Aids 

576 

Develop  New  Instructor  Guide 

81 

Validation  -  Preset'ts  Lesson 

475 

Validation  -  Revises  Lesson 

325 

Proofread  Final  Product 

553 

CBI/Slide  Conversion* 

268 

19% 

Video  Tape  Production 

251 

6% 

3706 


*Hours  reflect  the  tine,  based  on  the  ETV  document,  required 
to  prepare  and  supervise  a  contract  to  produce  the  low 
level  CBI  effectively  used  in  the  ACC  project.  The 
percentage  shown  includes  the  544  hours  actually  taken  by  the 
project  staff  to  develop  the  ACC  CBI. 


Appendix  C 


Methods  for  Determination  of  Wit-hin-Govemment  and  Contractor 

Hourly  Rates  (FY  89  Dollars) 


Contractor 


Total  Project  Cost  in  Dollars 

Contractor  Rate  - - - - - - -  $62. 74/hr 

Total  Contractor  Staff  Hours 


Within~GovernTnent  (based  on  the  monthly  pay,  allowances  and 
benefits  of  a  captain  with  8  years  service  and  taken  from 
standard  pay  tables  and  U.S.  Army  Finance  and  Accounting 
Center  figures  on  equivalent  compensation  and  the  TRADOC 
DCSRMs  ATRM-159  Report) . 


Within-Government  Rate  *  [Base  Pay  +  BAQ  w/Dependents  + 

BAS  +  Indira r'.  Benefits  (Retirement,  Medical,  Death  and 
Survivor,  f;ccia,l  Security,  Federal  Tax  Advantage)  ]  /Hours 
Worked  Per  Month]  +  23.5%  for  Base  Operations  *  $31. 91/hour 


Appendix  D 

Resident  Training  Cost  per  JSrjiduac 


Pirggt-  costs 

Instruction 

12,550 

Overhead 

3,632 

Student  Costs 

18,447 

Indirect  Costs 

Base  Operations 

5,921 

Total  40,550  (7  modules)** 

*14  week  EOAC-RC  conducted  at  Fort  Belvoir,  VA 
**FY  85  data/FY  89  dollars 
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